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1. Introduction 

The birational geometry of the moduh spaces of curves A4g and 
t;;-j- ■ of principally polarized abelian varieties (ppav) Ag has been studied 

0\ . extensively, with results for large enough g starting with Harris and 

^ : Mumford and Eisenbud and Harris' [HM82, Har84, EH87] proof that 

J^g is of general type for g > 23 (followed by Farkas' proofs that 
(^ ■ the Kodaira dimension of A^23 is at least 2 [FarOO] and that A^22 is of 

general type [Far09]), and Tai's [Tai82] and Mumford's [Mum83] proofs 
that Ag is of general type for f^ > 9 and g > 7 respectively. 

A more precise question is to describe the effective cone of Aig {Aig 
^ ■ is of general type if the canonical class is in its interior). The slope 

^ . of the effective cone o Aig for small genus was described by Harris 

and Morrison [HM90], who, led by their results, conjectured that the 
Brill-Noether divisor has minimal slope. The minimal slope of Ag (and 
its relation to Aig) for f? < 4 was described by the second author in 
[SM92]. For the first non-classical cases, it turns out that the minimal 
slope of Ai is given by the Schottky form — the divisor of the locus of 
Jacobians — of slope 8 (see also [HH04]), while recently the effective 
slope of Aig has attracted a lot of attention, after G. Farkas and Popa 
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[FP05] disproved the slope conjecture of Harris and Morrison [HM90]. 
Despite the further work [Far06, Kho05], the slope of the effective cone 
of Aig is not known for g > 11. It is known by the work of Tai [Tai82] 
(see also [Gru09]) that the slope of the effective cone of Ag approaches 
zero as g — ?■ oo. However, no examples of effective divisors of slope 
less than 6, on either Aig or Ag, are known for any g, while it is not 
known if 6 is a lower bound for the slope of effective divisors on Aig. 
(We also note that the nef cones have been studied for Aig — see 
[GKM02, FG03, Gib09] and for A'g — see [HulOO, HS04, SB06]). In 
the last few years G. Parkas and Verra, and also Ludwig, have studied 
the geometry of various covers and fibrations over the moduli spaces of 
curves [FarlO, FVIO, FV09]. 

In this paper we concentrate on the moduli space TZg of Prym curves, 
the subject of a recent survey [Farll]: this is the moduli space of pairs 
consisting of a smooth genus g algebraic curve X together with a line 
bundle 77 on X such that rj'^ = Ox while rj 7^ Ox- Such a data defines 
an etale double cover of X, and the associated Prym variety is an 
element of Ag^i — thus we have a morphism p : TZg — > Ag^i. 

One then wants to extend this morphism to a suitable compacti- 
fication. Indeed, if one takes a partial compactification T?.^"'"* of TZg 
obtained by adding stable curves with one node, and partial compact- 
ification A^'^i of Ag-i obtained by adding semiabelic varieties of torus 
rank 1 (i.e. for which the normalization is a P^ bundle over an abelian 
variety, see [Mum83]) the Prym map can naturally be extended to a 
morphism 7?.^"''* — )■ A^Zi- Taking an actual compactification is trick- 
ier, as the structure depends on which toroidal compactification of Ag 
is taken. Alexeev, Birkenhake, and Hulek [ABH02] studied the exten- 
sion to a map to the second Voronoi toroidal compactification. Note, 
however, that no matter what toroidal compactification Ag^i of Ag-i, 
the morphism on the partial compactification (without defining it on 
TZg \ Wg"""^^ gives a rational map p : IZg --■> Ag-i. 

In general the Picard group of an arbitrary toroidal compactification 
Ag may be very large, and is not known (see [HS04] for the discussion 
in genus 4). However, for the perfect cone compactification the Picard 
group (over Q) has rank two, and is generated by the class L of the 
Hodge bundle, and the class D of the (irreducible in this case) boundary 
divisor. We note also that the perfect cone compactification is Q- 
Cartier, as modular forms of weight k are a line bundle with divisor 
class kL, and cusp forms of weight k are a line bundle of class kL — D, 
and thus any class aL — bD is their linear combination. 
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From now on we denote by Ag the perfect cone toroidal compactifica- 
tion of the moduli space of principally polarized abelian varieties, and 
consider the rational map p : TZg ---> Ag-i. 

In this paper we use the Schottky-Jung relations to compute the 
pullback under p of the theta-null divisor and thus compute the pull- 
back map p* on divisors, and use this to bound the slope of the effective 
cone of A5. Our main result is theorem 5 computing the map p* on 
divisors, and the following bound for the slope: 

Theorem 1. The minimal slope s of effective divisors on A5 satisfies 

5 4198 

7+->s>7+ (i.e. 7.7142 ...> s> 7.6696 . . .). 

7 - - 6269 ^ - - ' 

Recall that the slope of a divisor aL — bD is defined to be a/b. 
The new statement in the theorem is the lower bound for the slope. 
The upper bound is provided by the Andreotti-Mayer divisor Nq (see 
remark 6). Our result is thus the lower bound for the slope of effective 
divisors (or of Siegel modular forms) in genus 5, where no such bound 
was known a priori. 
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2. The moduli of Prym curves and the Prym map 

The moduli space of curves Aig admits a Deligne-Mumford compact- 
ification Aig, the Picard group of which is generated by the classes Ai 
(the determinant of the Hodge bundle), 60 — the closure of the locus 
of irreducible nodal curves, and Si — the closure of the locus of nodal 
curves whose normalization has two components of genera i and g ~i. 

The moduli space TZg of Prym curves admits a compactification TZg 
— see [Farll] for its history and details — which is a branched cover, 
which we denote n : TZg — )> Aig. The Picard group PicqiTZg) was 
described by Farkas and Ludwig in [FLIO], whose notation and con- 
ventions we follow. It is generated by the classes 7r*Ai (which by abuse 
of notations we denote Ai), classes 6'q, 6q, (5™™, and various components 
of the preimages of 6i that will not be important to us. 
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/^From the point of view of the moduh of ppav, into which Aig em- 
beds by the ToreUi map to Ag, IZg embeds to TZAg. We recall the de- 
scription of the boundary components of TZAg and of TZg from [Don87, 
sec. 2.3]. Indeed, consider the full level two cover Ag{2), from which 
TZAg can be obtained by taking a quotient by the stabilizer G^ of a 
given two-torsion point 7] within the symplectic group Sp(2(yf,Z/2Z): 

Gr, := {7 e Sp(2^, Z/2Z) \^T] = r] mod I?^} 

Then we have the covering maps Ag{2) — )■ TZAg — )■ Ag. The bound- 
ary components of .4.3(2) are indexed by non-zero two-torsion points /i 
on the ppav (or, correspondingly, on the Jacobian of the curve if one 
considers the boundary components of M.g{2)). The stabilizer Gr, acts 
on the set of two-torsion points /i G (Z/2Z)^^ with three orbits, corre- 
sponding to whether fi ■ t] = 0, fi = rj, and /i ■ 77 = 1, respectively (here 
we view the points fi and 77 in (Z/2Z)^^ endowed with the symplectic 
pairing). For each of these three orbits, the union of the boundary di- 
visors of Ag{2) corresponding to fi lying in this orbit is invariant under 
Gr, and thus descends to a union of boundary components of TZAg. It 
turns out that in fact each of these 3 is an irreducible boundary com- 
ponent of TZAg, which we thus denote (5q, (5g , (5™™" correspondingly. By 
abuse of notation, following Donagi we also denote the same way the 
boundary components of TZg. 

We refer the works of Donagi [Don92] and Donagi and Smith [DS81] 
for the details on the structure of the Prym map for g = Q, the case of 
most interest to us. 

The map vr branches to order two along 5™"^, and is unramified on 
any other divisor. Recalling that the degree of tt is equal to 2^^ — 1, 
and counting the number of such fi in each case, we can thus record 
the pullbacks and pushforwards under n as follows (this is of course 
well-known, see [FLIO]) 

Lemma 2. The pullback and pushforward under n can be computed as 
follows: 

vr*(Ai) = Ai; 7r*(5o) = (5^ + 5^' + 25o™'"; 

7r,(Ai) = (2^^ - l)Ai; 7r,((5^) = {2'^-' - 2)6o; 

ttM) = So; MS'o^n = 2'^~%. 

We now recall that 77 defines an etale double cover X of a curve 
X of genus g, and the connected component of zero in the corre- 
sponding norm map of Jacobians Jac(X) — )■ Jac(X) naturally car- 
ries twice a principal polarization, and thus defines a Prym variety 
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Prym(X, r]) G Ag^i- This gives a morphism p : TZg — )> Ag-i, the Prym 
map, which Friedman and Smith [FS82] proved to be generically finite 
onto the image for (? > 6, and the fibers of which in lower genus were 
the subject of a lot of research [DS81, Don92, Iza95]. The Prym map 
can be extended to a rational map p : IZg --■> Ag-i- The geometry, and 
in particular the indeterminacy locus of such an extended map, for the 
case of the second Voronoi compactification, were studied in [ABH02] . 
Still, since p defines a morphism from the partial compactification T^^"*"* 
to the partial compactification .4.^"'^ , for any toroidal compactification 
— in particular for the perfect cone compactification — the indetermi- 
nacy locus is of codimension at least two, and thus the pullback map p* 
on divisors is well-defined. We recall (see [Tai82, Mum83, HulOO, HS02] 
for more details) that the Picard group PiC(Q(^g_i) is generated by the 
class L of the Hodge bundle and the class D of the boundary. 

For future use, recall the structure of the Prym map p along the 
boundary components of TZg. First, for a generic curve C = {Ci,p) U 
{C2,q)/{p ~ q) in 6i, we note that the Jacobian does not depend on 
the points p and q (the components 6i of dAig are contracted in the 
Satake compactification). Thus for any i > the map p contracts 
n~^{Si) (more precisely, its open part, where it is defined) to a locus of 
codimension at least 2 in Ag-i, for any g > 3. 

For the boundary components 6q, 6q, (5™™" the situation is more del- 
icate, and we recall it following [Bea77, Don87, Don92, DS81, Don88, 
Iza95], see also [Farll]. Let (C,p, q)/{p ~ q) for C G -Mg-i be a general 
point of 6q. Then for a general point of 6q lying over it the double cover 
is the Wirtinger double cover, and the corresponding Prym variety is 
simply the Jacobian of C, independent of the points p and q. Thus p 
maps (the open part of) 6q onto the locus of Jacobians J^g-i C Ag-i. 
For genus g > Q this locus is of codimension more than 1 in the Prym 
locus, i.e. the map p contracts 6q. The double cover of a generic point 
of 5™™" is the so-called Beauville admissible double cover, i.e. a double 
covers of C branching at the points p and q, and the corresponding 
Prym is an abelian variety. The branching order two here comes from 
the fact that the points p and q appear symmetrically in the construc- 
tion. 

The double cover corresponding to a generic point of 6q is inadmis- 
sible. This is to say that the corresponding double covering curve will 
have geometric genus 2g — 3, and two non-separating nodes. Thus the 
corresponding Prym variety will no longer be abelian — rather it will 
be the semiabelic variety of torus rank one, obtained by compactifying 
a C* bundle over a Prym of C, where the line bundle corresponds to 
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the difference p — q under the Abel-Prym map. Thus for a fixed curve 
C one gets a two-dimensional family of such semiabelic varieties corre- 
sponding to S'^{C)/i, where i is the involution on the double cover C 
of C. A dimension count then shows that for g > 7 the space of such 
Pryms is two-dimensional over the locus of Pryms of dimension g — 2, 
and thus of total dimension 3g — 1, thus forming a divisor in the image 
p{TZg) (in fact an open part of this divisor is equal to the open part 
of the boundary dp{lZg) = piJZg) fl dAg-i). We note that p{Sq) is in 
fact also a divisor in A^ (in fact equal to the boundary D) for g = Q. 
Indeed, the map p : 7^5 — > Aa has fibers generically of dimension two, 
and for a fixed point in A4 there is a two-dimensional family of points 
in 7^5 mapping to it. For each such curve there is a two-dimensional 
family of pairs of points on it, and thus for a generic point in A4 we 
have a four- dimensional family in 6'q C TZq such that the abelian part 
of the corresponding semiabelic Prym is equal to this point in A^ (and 
the degree of this map 7^.5^2 — ^ ^^4 to the universal family of ppav is 
equal to 27, see [Don92, Iza95]). 

As a result of this discussion, we get 

Proposition 3. For genus g > Q the composition of the map p* on 

Picqi^Ag-i) together with projecting to the span of 6i and 6q is zero 
(i.e. the classes 5i and Sq do not appear in any pullhacks under p* of 
divisors on Ag-i). 



3. The fullback under the Prym map: the Schottky-Jung 

equation 



In this section we compute the pullback map p* : PicQ(^c,_i) — )■ 
PicQ(7?.g). From now on we restrict ourselves to the case oi g > 6, 
whence in view of the above only the classes Ai, 6q, and 5™™ can appear 
in the pullbacks p*L and p*D that we need to compute. Note that a 
generic point in (5™'" has a smooth Prym (we have p : (5™™" --->■ Ag-i), 
while a generic point of 6q maps to a point in the boundary of Ag-i. 
Thus we know a priori that 

(1) p*L = aXi - 6(5™™; p*D = 06'^ 

for some a,b,c E Q, and the goal is to compute these coefficients. The 
standard method would be to do some test curve computations, by 
choosing three curves in TZg and computing their intersections with 
Ai,5q, and (J™'", and the intersections of their images under p with 
L and D. However, this seems rather tricky, as it requires knowing 
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the explicit geometry of the Prym map. Instead, we compute analyti- 
cally the pullback of the theta-null divisor by using the Schottky-Jung 
proportionality. 

We recall that the Schottky-Jung proportionality, due classically to 
Schottky [Sch88] and Schottky-Jung [SJ09], and in its modern form 
to H. Farkas and Ranch [FR70], relates the values of theta constants 
of the curve and of the Prym. We recall that theta constants with 
characteristics e, 5 G (Z/2Z)^, written as strings of zeroes and ones, 
are defined as 



e 



Y^ exp (tt? (n + e/2)* (r (n + e/2) + 6)) 



n^la 



where r G l-ig is a point in the universal cover of Ag — the Siegel upper 
half-space of symmetric g^ g matrices with positive definite imaginary 



part. Alternatively we can think of m 



T£+de 



G (Z/2Z)29 as a point of 



order two on the ppav, and write this as 9m{T~)- A characteristic m is 
called even or odd depending on whether the scalar product £-(5 G Z/2Z 
is or 1, respectively. All odd theta constants vanish identically, and 
there are 2^~^(2^ + 1) even characteristics. 

For the Schottky-Jung relations, iirj is chosen to be r] 



then the relation is 



1 



(2) 



e 



o- 



const -9 



05 



{r)-e 



Oe 
15 



where r is the period matrix of the curve C, a is the period matrix of 
the Prym, and the constant is independent of the characteristic e, 5. 

In general for arbitrary 7] the Schottky-Jung relation can be obtained 
by applying modular transformations to the relation above. The result 
is as follows: for a suitable embedding j : {Z/2Zf^3-i) ^ {J^jTLf^ 
such that Im(j) C r^-*-, we have 

(3) e]X^) = 0(n) ■ const ■^j(")(^) ' ^j(n)+vi^), 

where is now an eighth root of unity depending on n (this situa- 
tion was studied in more detail in [Far89] for one particular choice 
of f], where 0(n) was derived explicitly). We refer to [Farll, GrulO] 
for history, more details, and further references on the Schottky-Jung 
proportionalities. 

We will now pull back the theta-null divisor divisor on Ag-i under 
the Prym map. Recall that the theta-null divisor is defined as 



(4) 



7null 



{reAg 



n 



9r, 



0} 



me{Z/2Z), 



2g 

even 
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and its closure in Ag has class 



(5) 



[0, 



null 



2^-2(2^ + 1)L - 2^3-^D 



which follows from the fact that in genus g there are 2^~^(2^ + 1) even 
theta constants, each of which is a modular form of weight one half, 
together with a vanishing order computation (see [Fre83, Mum83]). By 
using the Schottky-Jung proportionality (3) we compute the puUback 
of the sixteenth power of the theta-null divisor: 



(6) 87r*[2^„,u(^)] = 8 



{r,v)eng\ JJ e^r) 



m£ri^. 



standard choice of characteristic 7] = 
sists of all characteristics of the form 



(notice that the eighth root of unity in (3) is torsion and does not 
matter for divisor class computations over Q). 

The right-hand-side of the equation above is the zero locus of a prod- 
uct of 2 ■ 2^^^ (2^^^ + 1) theta constants. Indeed, for example for the 

'0 ... Ol 

1 ... 

'O a 

X /3 

a, /3 is even. Since each theta constant is a modular form of weight one 
half, we get for the pullback of twice the theta-null divisor, using (5) 
for (? — 1, 

2p*(2^-3(23-i + 1)L - 2^3-^ D) = 2^-2(29-1 + i)Ai + . . . 



, the set T]^^^^ con- 



where X is arbitrary, and 



It thus remains to compute the boundary coefficients of this pullback. 
Equivalently, this means computing the slope of the modular form, so 
we need to compute the vanishing order of the right-hand-side of (6) 
near various boundary components of TZg. Recall (see [vG84, Don87]) 
that boundary components themselves correspond to points /i of order 
two, and the components Sq and (5™™" correspond to the cases oi fi ■ r] 
being and 1, respectively (while fi ^ rj, which would be 6q). For an 
analytic computation, instead of fixing rj it is easier to fix /i, i.e. to 
choose a standard boundary component, and study the degeneration 
there. This means we study the vanishing order of the right-hand-side 

of (6) as the period matrix of the curve degenerates as r — )> I , , 

for some b G C^~^ and some r' G Tig^i. This corresponds to the case of 



the standard cusp /i 



A theta constant 6 



r m 



such a limit has vanishing order (approaches the generically non-zero 
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9 



(r')) if and only if x = 0, and has vanishing order 1/8 if x = 1 



(in which case the lowest order term in the Fourier- Jacobi expansion is 
(9(g^/*)) along the boundary of Aig. We refer to [GHll] for a related 
detailed discussion. Note, however, that since the map ir : TZg — )■ Aig 
has branching order two along (5™™, the local coordinate w transverse 
to 5™™" on TZg is the square root of the local coordinate q transverse to 
So on J^g, and thus the vanishing order on 6o needs to be computed 
in terms of g^^^ — i.e. will be double the vanishing order computed in 
terms of q. 

Thus to compute the vanishing order of the right-hand-side of (6) 
at the standard cusp we simply need to count the number of charac- 



teristics m G rj^cn with x 
corresponding to fi ■ rj = 1 



= 1, multiplied by 1/8. In the case of (5, 
we can choose for example 



ram 
; 



V 



1 




SO that 



n2(3-1) 



Ve 



X a 

p 



for arbitrary x and any a, (3 E {Z/2Z)tmn''' ■ Thus we have 2^~'^{2^^^ + 
1) characteristics in f]^^^ with x = 1, and the vanishing order here is 
g,29 (29 +i)/8^ Thus along (5™™, with coordinate w = q^l'^ transverse 
to the boundary, the vanishing order is 2^~*^(2^~^ + !)• 

Similarly, for the case of ^q, corresponding to /x- 77 = 0, we can choose 



T] 




1 



so that 



^e 



X ^ a 

y z 13 



for x,y,z E Z/2Z and a, (3 E (Z/2Z)^^^ such that altogether the char- 
acteristic is even, i.e. xy + a ■ /3 = 0. To compute the number of such 
characteristics with x = 1, we note that for x = 1 the parity is equal 
to y + a- (3. Thus if one chooses z, a, (3 arbitrarily (so there are 2 ■ 2^^"*^ 
such choices), there exists a unique y such that together with x = 1 
the overall characteristics is even. Thus the total vanishing order here 
is equal to 1/8 ■ 2'^9~^. 

Using these vanishing orders, we finally get 

Proposition 4. For any g > Q the pullback of the theta-null divisor 
under the Prym map is given by 

p*(2^nuii) = p*(2^-2(29-i + 1)^ _ 225-6^) 

= 2^-2(23-1 + i)Ai - 23-^(2^-1 + 1)5™™ - 2^3-65^. 

For the sake of completeness, we also note that the vanishing order 
computation can also be done along 6q, showing that the vanishing 
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order is zero there. Thus even for g < Q, when p does not contract 6q, 
this class would not appear in the pullback. 
As a corollary, we get our first main result 

Theorem 5. For any g > Q the pullback of the divisor classes under 
the Prym map is given by 

p*L = X^-^6^r; P*D = S'o. 

Proof. To prove the theorem we use the proposition above to determine 
the coefficients in (1). D 

4. Bounds on the slope of A^ 
We recall that the slope of an effective divisor E = aL — bD on 



Ag-i with a, 6 > is deffned to be s := a/b. If E is the zero divisor 
of a modular form /, then its slope is the quotient of its weight a 
and its vanishing order b at the boundary (so the slope is ffnite only 
if / is a cusp form). Similarly, the slope of an effective divisor E = 
aXi — 6o<^o "" X^ bi6i on Aig is defined to be the minimum of all ratios 
a/bi. Farkas and Popa [FP05] proved that if the first ratio a/ bo is 
sufficiently small, then the slope of E is equal to a/bQ. 

Remark 6. An upper bound for the slope of the effective cone (the 
minimal slope of effective divisors) on A^ in theorem 1 is provided by 
the Andreotti-Mayer divisor — the component of the locus of ppav 
with a singular theta divisor different from the theta-null [AM67]. We 
recall that its class was computed by Mumford [Mum83] in arbitrary 
genus: 

Kl = (^ + f - 2-^(2' H- 1)) i - (^ - 2^-«) D. 
Thus in genus 5 we have on A^ the following formula for its slope: 
[N^] = 108L - UD; s(Ar^) = ^ = 7 + ^ = 7.7142 . . . 

yielding the upper bound in theorem 1. 

To obtain a lower bound for the slope of effective divisors on A5, we 
consider the map n^p*. Combining theorem 5 and lemma 2 allows us 
to go from Ag-i to Aig-. 

Proposition 7. We have the following formulas for the map Tit.p* on 
divisors: 

n,p*L = 7r,(Ai - 5o™"/4) = (2^^ - l)Ai - ^'^-^do; 



THE PRYM MAP ON DIVISORS, AND THE SLOPE OF A U 

Remark 8. As a good consistency check for our formulas we can verify 
that the pullback of the theta-null divisor on Ag^i to TZg, pushed down 
to A4g, gives a multiple of the theta-null divisor. Indeed, we compute 

7i*P*i0nun) = 2^-3(2^^1 + 1) ((2^^ - l)Ai - {2^^-^ + 2^^"^ - 2^-')6o) 
= 2^-3(2^-^ + 1) ((2^3 - l)Ai - {2^^-^ - 2^'^)6o) 
= 2^-3(2^-^ + 1)(2^ - 1) ((2^ + l)Ai - 2^-^60) 
= (2^~i + 1)(2^ - 1) (2^-3(2^ + l)Ai - 229-6^0) 

The corollary of this computation that we are interested in is a lower 
bound for the slope of the effective cone of A5, proving the new inequal- 
ity in theorem 1. 

Corollary 9. For any effective divisor E = aL — bD on A5, with 
a,b > 0, its slope s = a/b satisfies 

4198 

s>7+ , i.e. s > 7.6696 .. . 

6269' 

Proof. Indeed, using the proposition we compute 

Tr,p*[E] = 7r4p*{aL - bD)) = (2^^ - l)aAi - {{2^''^ - 2)6 + 2^'~''a)6o. 

Since the map p : T^g ""^ A5 is dominant, the class 7r^,p*[E] is the 
class of an effective divisor on Aig (notice that the class of the strict 
transform oip^^{E) C TZq may of course not be equal to p* [E] , but since 
we have p*[E] = [p^^{E)] + Exceptional, we certainly know that p*[E] 
is the class of an effective divisor), and thus its slope is bounded below 
by the minimal slope of effective divisors on A^g- The minimal slope 
of effective divisors on Aie is equal to y, achieved by the Gieseker- 
Petri divisor (note that 6 + 1 is prime, so there is no corresponding 
Brill- Noether divisor). Thus we must have 

(2^9 - l)a 47 



22s-% + (225-1 - 2)6 - 6 
solving which for g = Q yields 

48081 „ 4198 „ 

s > = 7 + = 7.6696 . . . 

- 6269 6269 



D 



Remark 10. We also note a peculiar consequence of our computations 
for the case oi g = 7. Recall that the Kodaira dimension of Aq is not 
known. The slope of the canonical bundle s{K^) is equal to 7, and 
thus for the Kodaira dimension of Aq to be nonnegative is equivalent 
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to the divisor 7L — D being effective. We then compute in this case for 
the slope 

1(12^ 
sin^p*i7L-D)) = s(7-(2i4-l)Ai-(7-2io+2i3-2)5o) = 7+^^ = 7.4 . . 



However, the minimal slope of effective divisors on A^ 7 is achieved by 
the Brill-Noether divisor, which has slope 6 + g^ = 7.7. . .. Thus any 

effective divisor on Aq of slope at most 7 must contain the Prym locus 

p(7^7). 

Appendix, by Klaus Hulek 

In this appendix we will discuss the notion of slope for arbitrary 
toroidal compactifications A^°^ of Ag. Besides the perfect cone or first 
Voronoi compactification Ag = A^'^^^ discussed in the main paper, two 
other toroidal compactifications have been considered by numerous au- 
thors, namely the second Voronoi compactification •AY"'^ and the central 
cone compactification ^4^^^"^* which coincides with the Igusa compactifi- 
cation. 

The perfect cone compactification has the special property that the 
boundary is irreducible and thus, over Q, the group of Weil divisors 
coincides with the group of Cartier divisors, being generated by the 
Hodge line bundle L and the boundary D. For general toroidal com- 
pactifications the situation is much more difficult: the boundary is not 
necessarily irreducible (this already happens for the second Voronoi 
compactification in genus 4, where the Picard number is 3, see [HS04]), 
and thus one cannot expect that the variety A^^^ is necessarily Q- 
factorial. Nevertheless it is true that for the group of Weil divisors one 
has 

r 

where Dq is the closure of the boundary of Mumford's partial compact- 
ification, which is contained in all toroidal compactifications, and the 
Di,i > 1 are further boundary divisors. Recall that toroidal compact- 
ifications are determined by the choice of a suitable fan in the rational 
closure of the space of positive definite real {g x 5()-matrices and that 
the Di are in 1-to-l correspondence with Sp{2g, Z)-orbits of rays in this 
fan. Here Dq corresponds to the orbit of rank 1 matrices. Any irre- 
ducible effective Weil divisor H, which is not a boundary component, 
can be written in the form H = aL — boD^ — Y7i=i ^1^1 with a, hi > 0. 
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In analogy with the case of M.g we suggest to define the slope of H by 

s\ope{H) = . 

min|Oj| 

and the slope of A^""^ as the infinum of slope(iJ) for all effective H. 
The main observation of this appendix is 

Theorem 11. For every toroidal compactification we have 

slope(^^°'-) = slope(^f "^0- 

Proof. Recall that toroidal compactifications are normal and thus reg- 
ular in codimension 2. It is thus enough to consider the stack-smooth 
part of .4.*°'" where we have at most finite quotient singularities and 
where we can also disregard the difference between Cartier and Weil 
Q-divisors. Any irreducible divisor H which is not a boundary divisor 
is given as the zero-locus of a modular form F. We thus have to un- 
derstand the vanishing order of the form F on the different boundary 
divisors. In fact, we claim that the minimal vanishing order occurs on 
the boundary component Dq of Mumford's partial compactification. 
Such an argument was first used by Tai [Tai82, p. 426 ff] in his proof 
that Ag is of general type ioi g > 9. 

In order to prove this we first remark that the map from the par- 
tial compactification of the quotient of Siegel space Mg by the center 
of the unipotent radical of the normalizer of a cusp to A^°'^ is unram- 
ified [Tai82, Section 5] at a general point of each boundary compo- 
nent. Hence it is sufficient to consider the Fourier- Jacobi expansion of 
a modular form F{Z) with respect to the cusp associated to a, g' < g 
dimensional isotropic subspace of Q^^ (this cusp is unique modulo the 
action of Sp(2(7, Z)). This expansion is of the form 



f{Z) = Y,Os{r,w)e 



2TTitr{Sz) 



where g' + g" = g, r e Hg/, w E Mat(5f' x g",C) and z G Mat(5f" x 
(7", C). Here S runs through all {g" x 5(")-dimensional semi-integral, 
semi-positive matrices. The claim then follows from a result of Barnes- 
Cohn [BC], which says that the minimum value of tr(S'X), where X is 
an integral, positive or semi-positive matrix, is attained at some matrix 
of rank one, i.e. at X = x*x for some 7^ x G Z^ . In other words the 
vanishing order is minimal at the boundary divisor Dq. D 

Corollary 12. Theorem 1 holds for all toroidal compactifications of 

A. 
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Remark 13. We would like to point out that the proof of Theorem 
11 does not generalize to moduli spaces of non-principally polarized 
abelian varieties as in this case the integral structure of the rational 
closure of the space of positive definite {g x 5()-matrices is not the 
standard one. 
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